feat(engine): trusted-ingestion parse path + detection cap#37
Merged
Conversation
Lift Studio past the browser's ceiling for files the user opens (S3): - Ingestion caps are now a `Limits` value chosen by the source's trust level (ADR-0007). `parseTrace` keeps the browser-scale untrusted caps (10 MB / 10k); `parseTraceTrusted` raises them to 256 MB / 2M for opened files. The workspace opens files trusted; live capture (S5) stays untrusted. - A 500k-event trace parses, correlates, and stays viewport-bounded in the timeline — proven headlessly (a 500k-line JSONL parses to 500k events; the virtual window stays viewport-sized). - Failure detection is capped at 50k events: several rules are O(n^2) (they mirror the toolkit's algorithms), fine at browser scale but a load-time stall at 500k. Past the cap the trace is fully inspectable and the status bar shows "detection skipped (large trace)". The O(n) detection rewrite is tracked in #36. Closes #29
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Lift Studio past the browser toolkit's ceiling for traces the user opens — the M8
"beyond-browser capacity" differentiator — while keeping the strict caps for
untrusted input.
parser.Limitsvalue chosen by the source.parseTracekeeps the browser-scaleuntrusted_limits(10 MB / 10k) unchanged;parseTraceTrustedappliestrusted_limits(256 MB / 2M) for files the user explicitly opened. Theworkspace opens files trusted; future live capture (S5) stays untrusted.
at a time — no whole-file JSON tree), correlates, and the timeline's virtual
window stays viewport-sized. All on the load path is O(n) / O(n·sessions).
event count (they mirror the toolkit's algorithms — fine at 10k, a load-time
stall at 500k). Past
detection.max_events_for_detection(50 000) the workspaceskips detection: the trace still parses, correlates, and is fully
inspectable, and the status bar shows
detection skipped (large trace)insteadof a misleading empty result. The O(n) rewrite is tracked in perf(engine): O(n) detection rules for dataset-scale traces #36 (with the
contract-v1goldens as the bit-exactness safety net).Verification
native test -Dplatform=null— 69/69 pass (macOS; CI runs Linux too). Newcoverage: the trusted path parses a 500 000-event trace;
parseTraceWithLimitshonors explicit byte/event caps; the untrusted default still rejects >10 MB /
native check --strictclean;native buildproduces the ReleaseFast binary.Security note
This raises an untrusted-input bound only for trusted sources (files the user
opened), keyed to source, not globally. The untrusted
parseTracepath — the onefuture live-socket capture will use — is unchanged. The trusted cap is bounded
(256 MB / 2M), so a pathological trusted file still cannot exhaust memory.
Closes #29